1. Field of the Invention
The invention relates to a method and system for capturing, storing and retrieving digital image data and one or more multi-media digital data entities by means related to the geographical location at which the images were captured. More particularly the invention has embodiments which include hyper-links between the recorded image data and stored digital multi-media data entities in which at least a portion of the entities are stored in a compressed Portable Document Format (PDF). Encryption of the digital data as recorded may also be performed using the geographical position data as at least part of an encryption/decryption key to provide secure authentication of recorded data to a second party.
2. Previous Art
Many applications of image or picture recording associated with the recording location are known. Tsuchiya, U.S. Pat. No. 5,267,042 shows a image pickup device body having a position detection means to record the device body position and an object image at the same time onto the image recording medium. Tsuchiya discloses a recording medium as photosensitive film or a magnetic video tape recorder. Location data is captured on the same film or video image as geographical coordinates, or alternately is captured on an sequential or parallel associated image. Means are provided for converting the digital coordinate data to an appropriate image for recording on film or with the video signal. It would be an advantage to eliminate the conversion step from digital coordinate data to an image of the coordinates.
Tsuchiya also discloses generating signals from an azimuth sensor and subsequently carrying out A/D conversion of the azimuth sensor signals. The converted azimuth sensor signals are coupled to a photosensitive unit which records the azimuth along with the image on the photosensitive film. It would be an advantage to directly store the image and coordinate data in a digital format, thereby eliminating extra conversion steps.
In Honda, U.S. Pat. No. 5,296,884, there is disclosed a data recording camera which automatically records an object image along with data relating to place at every photo operation. Place names and other data related to place are recorded along with image data, at the time of recording the image, on an IC card, magnetic disk or optical disk by means of a location determining device, such as a GPS receiver.
The camera operator selects one or more place data from a predetermined place data list or storage device incorporated in the camera. The operator selects the desired place data by means of a plurality of switches. The selected place data, such as a place name, eg Los Angeles, is recorded along with the object image. The place data is either imposed on the image data or is included as another display element. The additional display element may include the place data as letter images or codes.
It would be an advantage to automatically select the place data as a function of the camera location, thereby saving the operator time and effort of multiple switch manipulations.
Honda also discloses searching of images based on recorded place data, by reproducing the recorded place data and/or date and time data and searching for a comparison with a predetermined criteria. When a match is found, the selected image, corresponding to the selected place and/or time and date data, is reproduced and displayed. It would be more convenient to minimize the delay caused by reproducing and searching and comparing a long sequence of place or date/time images, in order to select one or more recorded object images.
Gunthorpe, et al, in patent application PCT/AU92/00667, disclose a portable distance calculator method and apparatus. The distance calculator apparatus and method provides for locating and displaying a map of a golf course with the relative location and distance of a golf ball at successive locations from a subject pin or dog leg on the course. The claimed apparatus displays a map indicating the location of the golf ball and the distance to the pin or dog leg during the successive shots.
No showing is made of simultaneously recording and storing object images associated with the location of the golf ball or the player carrying the apparatus.
U.S. patent application Ser. No. 08/191,815 by Janky, et al, incorporated herein by reference, discloses a remote target locator apparatus which incorporates GPS map and target location and display capability. A potential target is illuminated by a thin beam of directed energy and viewed through an imaging device. The returned energy scattered from the target is received and processed to determine a distance value between the locator apparatus and the target.
A GPS antenna array on the locator is used to provide the attitude (ie slope, azimuth) of the pointing beam. A computation means, such as a miniturized single board computer, calculates the location of the remote target relative to the computed and known location and attitude of the locator apparatus. The target image and location coordinates are then transmitted to a selected remote receiver for bringing fire to bear on the target.
No image storage or processing other than transmission is contemplated or discussed in Janky. Also, no provision is made for relating target images to associated map features by geographic location coordinates. It would be advantageous to relate the acquired target to nearby features such as the proximity to sensitive structures or personnel which may be endangered by attack.
U.S. patent application Ser. No. 08/440,246 by Woo, "A satellite Navigation receiver with Map Graphics and Hypertext Driven by a PDF Database", incorporated herein by reference, discloses the use of specially compressed Portable Document Format (PDF) database files to store map graphic data with hypertext links between the location of a self contained navigation receiver, other points of interest and other stored data items such as text, graphics and images. The PDF format is described in Portable Document Format Reference Manual, Adobe Systems, Inc., 1993, Addison-Wesley Publishing Company, ISBN 0-21-62628-4. The PDF format permits great detail and high resolution map images and other multi-media entities to be stored with great efficiency. The PDF format enables compact storage, representation and display of graphical map data which is important for portable and hand held equipment in that it minimizes size, power and expense of memory required for storing and displaying graphical map images.
PDF files allow the use of computer files that are independnt of the appliation software and operating system to create it. Therefore, PDF files can be readily converted to and from "POSTSCRIPT".TM. and Apple Computer "QUICK-DRAW".TM. file types. Each PDF file contains a PDF document and other supporting data. Each PDF document has one or more pages which may contain text, graphics and images in a device and resulution independent format. A PDF document may also contain purely electronic representations such as hyper text links. Each PDF file further contains the version of the PDF specification used in the file and information about the important structures in the file.
A commercial product called "PDF Writer".TM. and computers running both the Apple MACINTOSH.TM. and computers running Microsoft "WINDOWS".TM. (e.g., GDI) that acts as a printer driver for POSTSCRIPT.TM. and GDI application programs. A printer driver ordinarily converts operating system graphics and text commands into commands that will be understood by a particular attached printer. Such drivers enbed commands into printer command streams for page printing. The PDF WRITER.TM. sends such command streams after emvdedding to a PDF file instead. The resulting PDF files are platform independendt, e.g., they may be freely exchanged between MACINTOSH.TM. and WINDOWS.TM. computers. PDF files are seven-bit ASCII and may be accessed by PDF viewing applications on any platform, e.g., Adobe ACROBAT EXCHANGE.TM. running on the MACINTOSH.TM.. The imaging model of the POSTCRIPT.TM. language is used by PDF to represent text and graphics. A PDF page description draws a page by placing "paint" on selected areas of a blank white page. Painted figures may be letter shapes, regions defined by combinations of lines and curves, or digitally-sampled images of photographs other or images. Such paint can be any color or shade. Any figure can be clipped to another shape, such that only parts of the figure within the shape will appear in the page. PDF uses marking operators similar to POSTCRIPT.TM. marking operators, but it is not a programming language, and so does not include procedures, variables and control constructs. As a result, applications can more efficiently and reliably locate text strings in PDF files, compared to POSTCRIPT files.
PDF files support industry-standard compression filters likd JPEG compression of color and grayscale images, CCITT Group-3 FAX, CCITT Group-4, Lempel-Ziv_Welch (LZW) and run length compression of monochrome images and LZW compression of text and graphics. Such compression is important to maximizing how much page information can be stored by any particular memory and the maximum communication rate needed for any particular connection.
It would be an advantage to incorporate efficient file compression, data storage and display techniques into geo-graphically referenced image capture and display apparatus.
U.S. Pat. No. 5,450,344 by Woo, and U.S. Pat. No. 5,422,814 by Sprague, et al, incorporated herein by reference, disclose GPS receivers with Data Ports for uploading and downloading of Absolute Position Information. This is important for providing means for storing and transmitting absolute geographical position data, such as map coordinates, between a GPS receiver and other related equipment, eg, computer storage media (disk, RAM, and the like). By this means, absolute geographical latitude and longitude position data may be related and/or referenced to other associated data items at the time of position capture. Retreiving or storing data items based on a reference to a geographical location is sometimes referred to as geo-addressing.
Geo-addressing generally refers to the absolute latitude and longitude of a geographical location which is related in some way to another object. There may also be associated with a particular geo-address, additional position data such as orientation or attitude (ie azimuthal, elevation and roll from some reference axes) of a line of sight from the specific geo-address to an associated target object and a range or distance between the specific geo-address and the associated object. The term geo-addressing in the present invention includes at least the latitude and longitude in addition to the above position factors.
Recorded digital data: encryption for authentication
In evidentiary proceedings such as photo evidence to be gathered for court testimony, it is important to authenticate the images (or other data) which are recorded at a scene for later use by the court. One method of authentication is to embed a secret or hidden message into the recorded image which is not accessable by an unknowing party. Steganography, from the Greek for `covered writing`, is a form of cryptography in which a first message to be hidden is concealed within another message or image. Steganography is briefly discussed in "Codes and Ciphers" by Fred B. Wrixon, Prentice Hall General Reference, New York, N.Y., 1992. One method of concealment expresses each pixel in an electronic representation of the image in digital form and then alters the least significant bit of a chosen number of pixels to contain an authentication message.
These methods usually rely upon some unchanging characteristic or combination of characteristics of an entity that seeks access to a protected system or a protected image for the purposes of changing or displaying some aspect or aspects of the image. The protection offered by these approaches might easily be compromised for an image transmitted over a communication channel that is continually or periodically transmitting information.
What is needed is an approach that relies in part on the constantly changing information associated with signals produced by a location determination system. The changing position/location/time information may be used to authenticate or verify the location and/or angular orientation and/or time of image formation of the entity that forms an image that is to be authenticated, Preferably, the recorded information relied upon for authentication should be accurately available only from the entity that formed the image or from a properly designated agent. Preferably, the approach should accept and work with confidential/encrypted signals and/or with signals that are available for use by any user with an appropriate receiver for a location determination system.
Cryptography in digital systems is discussed in some detail in the "Encyclopedia of Computer Science", 3rd Edition, Anthony Ralston and Edwin D. Reilly, Van Nostrand Reinhold, New York, N.Y., 1993, pps 369, 370. Cryptography is the science of transforming messages for the purposes of making the message unintelligible to all but the intended receiver of the message. Data encryption refers to the use of such methods in computer or digital data for the same reason, but also implies the additional goals of providing assurance to the receiver of the message that the message is not a distorted, altered or otherwise changed message or image, and/or allowing the message receiver to prove to a third party that the message is authentic.
The transformation used to encipher a message typically involves both a general method, or algorithm, and a key. While the general method used by a pair of correspondents (eg evidence gatherer and receiver) may be public knowledge, some or all of the key information must be kept secret. The process of transforming (enciphering or encrypting) a message is to apply the enciphering algorithm to the message, where the key is used as an auxiliary input to control the enciphering. the reverse operation (deciphering or decrypting) is performed similarly. The enciphered data is combined with a suitable deciphering algorithm and the corresponding deciphering key, by a suitable deciphereing means, to reveal the original message, image, or data.
Standard enciphering algorithms include simple substitution codes, transposition codes and the like. A provably unbreakable ciphering algorithm is the one-time pad, described, op cit. To encipher a 1000 bit message, however requires the use of a 1000 bit key that will not be used for any other message. Larger messages, e.g., a digital image, may be impractically large or expensive except for the most critical cases. Other ciphering algorithms are the Data Encryption Standard, (DES) approved by the National Bureau of Standards in 1976, various public key systems, such as those developed by Diffie and Hellman, and the Rivest, Shamir, Adleman (RSA cipher) developed in 1978, page 370 op cit.
Incorporating the time of image formation into the algorithm or key structure adds an additional dimension to the ciphering structure. Image formation time may be derived from a built in clock provided as part of the self-contained image recording or position determining system. Time may also be received from an external source, such as the standard time supplied by standard frequency transmission stations throughout the world, whose time signals, carrier and modulation frequencies are very precisely established by the various national laboratories. In the United States, details may be obtained from the Frequency-Time Broadcast Services, National Oceanic and Atmospheric Administration, Boulder, Colo. 80302.
For the purposes of image authentication, a specific recorded digial image or audio waveform stream may be thought of as a particular assembly of digital data; ie, a file, in the form of a sequence or array of bits or bytes.
Recorded Digital Data Formats
Digital data storage in a particular medium may be formatted in a proprietary manner or may be done in some standard format agreed to and published by a relevant standards body. Many standards bodies exist which publish standards for national or worldwide use, such as the ASCII, ANSII, IEEE, and the like. A relevant standard format for storing digital map data (eg. latitude and longitude) is described in the U.S. Department of the Interior, U.S. Geological Survey document FS-122-95, May 1995. The digital image is accompanied by a metadata file that complies with the Federal Geographic Data Committee approved Content Standards for Digital Geospatial Metadata (Jun. 8, 1994). This standard is particularly applicable to use in geo-referenced digital applications due to the definitions of standard attribute fields which can be used for storing specific kinds of information. This ensures wide usability among manufacturers and users of hardware and software.
The USGS standard includes fields which provide storage space for containing values of predefined attribute variables. These attribute values may then be used by application programs for controlling the use and display of map data.
It would be an advantage to capture geo-addressed object images using a hand held portable unit in which object image data and other data entities are stored and accessible by hyperlinks between portions of a PDF file.
It would be an advantage to capture geo-addressed digital audio waveform streams using a hand held portable unit in which digital audio waveform streams and other data entities are stored and accessible by hyperlinks between portions of a PDF file.
It would also be an advantage to provide an geo-addressed authentication or authentication means associated with such digital images.
None of the above systems provide rapid selection and display of geographically associated features or data entities related to a geographically related object image. Neither do any of these systems provide for image manipulation and combination other than location and attitude coordinates. Nor do they provide geo-addressed image authentication means.